Classification confidence weighted majority voting using decision tree classifiers

In this paper a novel method is proposed to combine decision tree classifiers using calculated classification confidence values. This confidence in the classification is based on distance calculation to the relevant decision boundary. It is shown that these values – provided by individual classification trees – can be integrated to derive a consensus decision. The proposed combination scheme – confidence weighted majority voting – possesses attractive features compared to other approaches. There is no need for an auxiliary combiner or gating network, like in the Mixture of Experts structure and the method is not limited to decision trees with axis-parallel splits; it is applicable to any kind of classifiers that use hyperplanes to cluster the input space.